Rate-constrained multihypothesis prediction for motion-compensated video compression

This article investigates linearly combined motion-compensated signals for video compression. In particular, we discuss multiple motion-compensated signals that are jointly estimated for efficient prediction and video coding. First, we extend the wide-sense stationary theory of motion-compensated prediction for the case of jointly estimated prediction signals. Our theory suggests that the gain by multihypothesis motion-compensated prediction is limited and that two jointly estimated hypotheses provide a major portion of this achievable gain. In addition, the analysis reveals a property of the displacement error of jointly estimated hypotheses. Second, we present a complete multihypothesis codec which is based on the ITU-T Recommendation H.263 with multiframe capability. Multiframe motion compensation chooses one prediction signal from a set of reference frames, whereas multihypothesis prediction chooses more than one for the linear combination. With our scheme, the time delay associated with B-frames is avoided by choosing more than one prediction signal from previously decoded pictures. Experimental results show that multihypothesis prediction improves significantly coding efficiency by utilizing variable block size and multiframe motion compensation. We show that variable block size and multihypothesis prediction provide gains for different scenarios and that multiframe motion compensation enhances the multihypothesis gain. For example, the presented multihypothesis codec with ten reference frames improves coding efficiency by up to 2.7 dB when compared to the reference codec with one reference frame for the set of investigated test sequences. Keywords— Video Coding, Motion-Compensated Prediction, Rate-Constrained Motion Estimation, Multihypothesis Motion-Compensated Prediction, Linear Prediction, Multiframe Prediction, Entropy-Constrained Vector Quantization